Intelligent process automation

If they can train you to do it, eventually they will train a computer to do it. However, technology should enhance human expertise not replace it.

What is Intelligent Process Automation?

Intelligent process automation (IPA), sometimes also called cognitive automation, is the use of automation technologies – artificial intelligence (AI), business process management (BPM), and robotic process automation (RPA) – to streamline and scale decision-making across organizations.
IPA in its full extent encompasses five core technologies:

Robotic process automation (RPA): a software automation tool that automates routine tasks such as data extraction and cleaning through existing user interfaces. The robot has a user ID just like a person and can perform rules-based tasks such as accessing email and systems, performing calculations, creating documents and reports, and checking files. RPA helped one large insurance cooperative to reduce excess queue procedures affecting 2,500 high-risk accounts a day, freeing up 81% of FTEs to take on proactive account-management positions instead.
Smart workflow: a process-management software tool that integrates tasks performed by groups of humans and machines (for instance, by sitting on top of RPA to help manage the process). This allows users to initiate and track the status of an end-to-end process in real time; the software will manage handoffs between different groups, including between robots and human users, and provide statistical data on bottlenecks.
Machine learning/advanced analytics: algorithms that identify patterns in structured data, such as daily performance data, through “supervised” and “unsupervised” learning. Supervised algorithms learn from structured data sets of inputs and outputs before beginning to make predictions based on new inputs on their own. Unsupervised algorithms observe structured data and begin to provide insights on recognized patterns. Machine learning and advanced analytics could be a game changer for insurers, for example, in the race to improve compliance, reduce cost structures, and gain a competitive advantage from new insights. Advanced analytics has already been implemented extensively in leading HR groups to determine and assess key attributes in leaders and managers so as to better predict behaviors, develop career paths, and plan leadership succession.
Natural-language generation (NLG): software engines that create seamless interactions between humans and technology by following rules to translate observations from data into prose. Broadcasters have been using natural-language generation to draft stories about games in real time. Structured performance data can be piped into a natural-language engine to write internal and external management reports automatically. NLG has been used by a major financial institution to replicate its weekly management reports.
Cognitive agents: technologies that combine machine learning and natural-language generation to build a completely virtual workforce (or “agent”) that is capable of executing tasks, communicating, learning from data sets, and even making decisions based on “emotion detection.” Cognitive agents can be used to support employees and customers over the phone or via chat, such as in employee service centers. A UK auto insurer that uses cognitive technology saw a 22% increase in conversion rates, a 40% reduction in validation errors, and a 330% overall return on investment.

✓ Automation of 50 to 70% of tasks…
✓ which has translated into 20 to 35% annual run-rate cost efficiencies …
✓ and a reduction in straight-through process time of 50 to 60% …
✓ with return on investments most often in triple-digit percentages.

Our experience shows that the promise of IPA (double-digit or even triple-digit same-year) is real if executives carefully consider and understand the drivers of opportunity and incorporate them effectively with the other approaches and capabilities that drive the next-generation operating model.

Key differences	RPA	IPA	Hyperautomation
Tools or technologies required	Performed by RPA automation tool	Performed by multiple machine learning modules and automation tools. Custom development required	Performed by multiple machine learning, packaged software (Process Mining, No Low code apps, analytics) & automation tools
Maturity level	Very matured, sophisticated, fast and easy to implement	Scaling. Sophisticated AI-based process automation with cognitive ability	Transforming. Sophisticated AI-based process automation with cognitive ability and can loop humans in the process
Outcome	Efficient operations	Efficient complex operations	Smart and efficient operations
Scope / Coverage	Rules based, routine repeat mundane tasks	Higher-function tasks that require some level of reasoning, judgment, decision, and analysis	All-ecompassing: "everything that can be automated will be automated"
Governance approach	Process-first	Process-first	People-first and Process-first
Implemented by	IT, citizen developers	IT	IT, democratization of Automation Development

What might IPA look like in action?

Intelligent automation simplifies processes, frees up resources and improves operational efficiencies, and it has a variety of applications. For example, an automotive manufacturer may use IA to speed up production or reduce the risk of human error, or a pharmaceutical or life sciences company may use intelligent automation to reduce costs and gain resource efficiencies where repetitive processes exist. An insurance provider can use intelligent automation to calculate payments, make predictions used to calculate rates, and address compliance needs. Let’s take an insurance company where a human claims processor pulls data from 13 disparate systems to provide a “business as usual” service. With IPA, robots can:

✓ replace manual clicks (RPA)
✓ interpret text-heavy communications (NLG)
✓ make rule-based decisions that don’t have to be preprogrammed (machine learning)
✓ offer customers suggestions (cognitive agents)
✓ provide real-time tracking of handoffs between systems and people (smart workflows)

Original processor

Case Studies

Creating an employee helpdesk with Intelligent Automation

Case study 1, for NLP: users can make natural language requests to a bot in Slack, which are recognized via a natural language processing tool like Dialogflow to analyze the intent of the input. Then, the bot will search ServiceNow’s knowledge base to retrieve the best answers to the question asked and post the results back in Slack. This process feels very natural to the employee and eliminates the need to speak with a Subject Matter Expert every time someone has a question.

Case study 2, for Finance: in the case of a banking system evaluating transactions for fraud, data is pulled from accounts via RPA to be fed into an AI algorithm. The AI engine evaluates whether the transactions match the pattern with which the customer typically spends their money and whether that customer is breaking any laws by funding illegal activities.

Case study 3, for ML and virtual machines: an example of intelligent automation would be using ML to analyze historical and real-time workload and compute data. An intelligent automation platform could then manage workloads to optimize runtimes and prevent delays, while provisioning and deprovisioning virtual machines to meet real-time demand.

Schema eCommerce AI generative

Case study 4, for e-Commerce: the objective is to allow the search for products with natural language to get closer to the personal shopper experience and provide product advice. The technical stack being used is comprised of Langchain, OpenAI, Nextjs, AWS, Postgresql, Supabase. The technical challenges are to create the best prompt to get the most relevant answers possible while automating the creation of embeddings from product descriptions and storing them in Supabase. We could imagine intelligent automation could also capture all prompts, export them in a standard structured format like JSON, set up a cron job to replicate the batch processing log files on a Linux server to a central server, or alternatively transport real-time visibility logs using an API, and store them on S3 for being processed, refined and being used for an analytical or monitoring platform.

Case study 5, for debugging or test-driven development (TDD): when a product is scaled, the code and infrastructure could become exponentially more complex and bugs started to appear. A full-time tester is great for obvious bugs but isn't always accurate from rare situations. Writing automated functional tests (e.g. intended for large-scale data science pipelines) as the unit level is the best option. They function as sensors that are run automatically at frequent intervals to check if features continue working as expected with adjacent functions even after when both have been modified.

Case study 6, for log monitoring:

Long Monitoring Automation

Issues

Too much noise: every application, service, device, and program generates logs at a near-constant rate. With so much data, it can be difficult to catch and resolve critical events.
Too many log types: not every component application or system generates logs in the same format. This makes parsing and searching log events from multiple sources both time-consuming and challenging.
Constant fire fighting to keep ahead: because logs are generated in enormous volumes, monitoring them manually is a near-impossible challenge. With such large volumes of data, it’s easy to miss trends and recurring problems.

Solutions

Scan events as they’re generated: obtain real-time insights into applications and systems.
Catch problems early: identify trends or changes in application or server performance and initiate troubleshooting before end users are affected.
Set custom alerts: receive notifications right when an issue occurs or generate daily summaries of events.

Case study 7, for mitigating the risk of bias:

IPA poses certain risks, including the risk of bias. Bias can occur when AI systems are trained on data that is not representative of the population, leading to decisions that are unfair or discriminatory. To mitigate these risks, organizations should be aware of the potential sources of bias in their data and take steps to address them. This can include using diverse data sets, implementing bias detection and correction algorithms, and involving a diverse team of experts in the development and deployment of AI systems. Additionally, organizations should establish robust monitoring and auditing processes to ensure that their systems are operating in a fair and unbiased manner.

Applications & Benefits of Intelligent Automation

IPA can be applied in a wide range of industries, the following provide examples of how IPA is being used in a variety of industries to improve efficiency, reduce costs, and enhance the quality of services.

Key benefits:
Intelligent automation platforms provide many benefits across industries as a result of the use of large amounts of data, precision of calculations, analysis and resulting business implementation.

· Reduce costs by augmenting the workforce and improving productivity: Automating systems and processes and using data and analysis to ensure accuracy can accelerate production. IA also provides the ability to scale quickly without increasing risk, compromising quality, or putting a strain on the existing workforce. Business leaders are reaping the benefits of this capability through higher production yields and improved ROI.
· Improve accuracy through consistent processes and approaches, which enhances quality: The strength behind intelligent automation is the use of artificial intelligence to drive decision-making and bring a consistent approach to repetitive tasks.
· Improve customer experience: Providing a higher quality, more reliable product to market faster, or getting answers to queries faster (or immediately) provides a richer, more positive experience for the customer, and thereby a competitive advantage for the company.
· Address compliance and regulations with confidence: Many industries are required to adhere to specific regulatory policies. Intelligent automation can leverage its task automation features to prove a more consistent approach to compliance.

RPA & AI

Difference between RPA and AI

Moving beyond RPA

The missing link in intelligent automation

Is Generative AI, such as GPT, suitable for IPA projects?
Generative Pre-trained Transformer (GPT) is becoming increasingly popular for natural language processing tasks such as text generation, question answering, summarization, and machine translation. The GPT model is composed of a deep neural network that is pre-trained on a large corpus of text, allowing it to learn the structure of language. This pre-training enables the model to generate human-like text and provides it with a strong foundation for further fine-tuning on specific tasks. GPT is a powerful language model that can be used for a variety of natural language processing (NLP) tasks, including IPA. GPT can be trained to classify text data into different categories, such as customer service requests, invoices, or legal documents. It can also be used to automatically summarize long documents or email threads, generate text in a certain format or style, analyze sentiment expressed in text, and generate human-like speech. These capabilities can be used to automate and streamline processes such as customer service, legal document generation, customer sentiment analysis, and more. Definitely, GPT can be used for IPA to automate and streamline business processes by leveraging its NLP capabilities in combination with other technologies.

How to get started on your IPA transformation

IPA does not require a significant infrastructure investment since it addresses the presentation layer of information systems. RPA software, for example, sits on top of existing systems, enabling it to be implemented to achieve rapid returns without changing the IT back end. In some cases, companies can get RPA systems up and running—and delivering value—in as little as two weeks.

In our experience, the following steps are the most important in driving successful IPA transformations at scale:

Rapidly align on IPA’s role in operating model: Any effective IPA initiative must be grounded in a clear understanding of the overall strategy of the business and the role of the next-generation operating model in helping to achieve it. That requires a clear articulation of the target end state and the journeys to focus on to reach it. Such clarity allows business leaders to evaluate and align on the approaches and capabilities to implement to drive the operating model. In many cases IPA has an important—even dominant—role in driving the change, but its greatest value comes when companies understand how it can work with the other capabilities and approaches in the operating model. Automation is coming, and now is the time to define the art of the possible and apply it strategically where it makes most sense.
Design around the full portfolio of IPA solutions to maximize impact: Organizations should not dabble with a few IPA technologies. The world moves too quickly for that approach to work effectively. The full impact comes when IPA technologies work together. Organizations need to envision and implement holistic optimization programs to maximize return on investment. Though it is easier and faster to implement automation projects in silos, such an approach is inherently flawed. By themselves, individual technologies are insufficient to capture value. Instead, fundamental process redesign is required to transform the way a group works. A detailed roadmap for implementation should be created to identify all automation-enhancement opportunities and allow businesses to sequence IPA initiatives by balancing their impact with the feasibility of scaling solutions from initial use cases. Start your IPA journey by rapidly creating an overview of current tasks and the resources and capabilities needed to carry them out. Then deploy an experienced ring-fenced incubator team to redesign processes and group workflows based on a deep understanding of lines of business and IPA capabilities.
Build a rapid minimum viable product (MVP): Even though it’s important to design for a full IPA portfolio, it can be daunting to start working on everything at once. Many executives have been burned by promising complex data-warehouse projects, some of which have taken up to a decade to complete and have run vastly over budget. As with other digitization efforts, it’s better to select—with a bias to speed and impact—an end-to-end process or customer journey to redesign and enhance using IPA, and then work to launch an MVP, the most stripped-down version of the product that can still accomplish the task. In this way, you can quickly test what works and what doesn’t and make changes accordingly. IPA can deliver tangible value in weeks rather than years in the form of fewer errors and less “busy work” for back-office employees. The rapid returns from early pilots help to secure support from stakeholders and executive sponsors for a much deeper program to harness the potential achievable from a full IPA transformation.
Build momentum and capture value: Any IPA implementation should combine quick wins with larger longer-term developments. The detailed roadmap should be rooted in a fundamental process redesign that sequences automated modules for production and reimagines the way groups should work to capture value. Every product line in insurance, for example, has a different degree of potential for standardization and automation, and needs to be examined and sequenced. Look at time-intensive processes in sales, underwriting and pricing, policy administration, claims and finance, and accounts, and start with a clean sheet when deciding how they will work in future.
Embed lasting capabilities to achieve sustainability: One successful way to sustain value creation is by creating a center of excellence (CoE) to govern the transformation and support the rapid deployment of IPA solutions through capability building, certification and standards, vendor management, and the creation of a library of reusable solution patterns. Such a CoE should be centrally located and can be fairly small in size because it can call on existing lean or process-optimization CoEs, while business ownership and execution should sit in the lines of business or in digital factories. Systematic controls need to be in place, and organizations should embed critical business-analysis and digital skills in lines of business so that they can own the process. They also need to redesign organizational structures to capture value, establish a future-state operating model to scale up their IPA initiatives, create blueprints for future structures to capture impact and embed new capabilities, and offer training and workshops to explain why the automation of manual processes will free up teams to focus on more creative activities.

Connecting Intelligent Automation and innovation in business

A CIO plan for becoming a leader in IPA

Intelligent Process Automation (IPA)—a set of technologies that combines process redesign, process automation, and machine learning—is rapidly reshaping the global economy, with significant gains for organizations that adopt it at scale. Some companies across industries have already been able to automate 50 to 70% of tasks with return on investment generally in triple-digit percentages. While people often focus on the cost savings, IPA also provides significant other benefits, including speed, precision, and improved customer service. But for companies to get the full value of IPA, IT will need to play a leading role. The track record of early adopters clearly demonstrates that IPA projects carried out without the active participation of IT are likely to fail. For CIOs to play a guiding role in IPA, they need to develop a core of expertise and experience developed by implementing IPA programs within IT. And it’s important to do so quickly. If CIOs don’t support automation across the company, then business executives will start building their own shadow IT organizations or working with external vendors. However, many IT executives struggle with successfully implementing IPA processes. The most frequently stated reasons are:

The higher complexity of IT compared to a business process
Difficulty in understanding the economics of IPA and a lack of clarity on how to best capture the benefits
Inconsistent and fragmented tools that make IPA hard to scale
The misconception that IPA is an advanced lever requiring massive process re-engineering before embarking on an automation journey

How can CIOs succeed? We have found that there are four key steps on the IPA journey that need to be mastered:

↓ Step 1: Assess the value potential at a high level

↓ Step 2: Drill down to specifics to understand which use cases are best suited for implementing IPA

How to implement IPA can vary significantly across identified activities and often requires digging into root causes for issues, untangling complex systems, and developing a clear understanding of how to approach implementing IPA to get the value. In some cases, we’ve seen businesses use specific IPA tactics to help unlock the necessary insights.

Responding to incidents: Understanding how to go about automating incidents starts with identifying which of them are most suited to automation, which can be challenging. While incidents are well documented, they are also numerous—large IT organization can easily generate a million tickets a year—and the root cause of each is often not readily apparent. “I don’t receive emails” does not necessarily indicate an email-program issue; it may simply mean “I lost my password.” Often businesses will try to automate incident responses without being clear about the “why,” resulting in poor outcomes. Specific text-mining tools that read ticket descriptions in detail and derive the necessary insights can address these complexities. Using this approach, we have been able to define ~50 different ticket types and divide them into IPA categories:

Automatable (as an example, 80% of “reset password” incidents can be automated)
Requires machine learning
Highly cognitive/manual

The output of this analysis should be a prioritized list of incidents to automate and which IPA element to use for each.

Conducting planned activities: While most IT groups have industry-standard tools for infrastructure management, we have seen that the complexity of configurations means that IT isn’t getting as much value from them as they should. A high degree of customization, adjustments because of mergers, and specific user requirements mean that significant manual labor is required to manage the systems. For example, despite the broad usage of application-monitoring tools (like Prometheus) and infrastructure-monitoring tools (like Zabbix), application support teams are often unable to act quickly and effectively on the logs generated because there are often too many of them, generated for a variety of reasons. The result is companies aren’t clear about how to go about implementing IPA. In this case, a machine-learning (ML) bot can help make sense of the complexity because it can be trained to learn the reasons behind a given alert and then recommend— or even make—better decisions about what action to take.

Delivering new applications: However, many CIOs fall into the trap of simply focusing on reducing manual labor, which limits the full value potential of IPA. More accurate and faster application delivery requires designing a new IT operating model, with an emphasis on agile and DevOps. Reviewing the entire process to understand how to make most effective use of agile and DevOps can lead to completely different approaches and ways of working. Some of those new ways of working can be enabled by IPA. Automating testing, for example, allows teams to iterate more quickly; creating a self-serve model for automated server provisioning allows operations to be more responsive. One major US life insurer approached this issue by developing a phased strategy for IT infrastructure automation. It started with developing a DevOps model for how infrastructure and operations teams could work together. The team then cooperated on building out a comprehensive IPA program supported by a relevant set of APIs that enabled the team to access varied sources of data. As it learned how to manage this approach, it migrated relevant parts of the infrastructure to the cloud to increase flexibility. The result of this effort was that the infrastructure organization, which originally consisted of ~1,400 full-time employees (FTEs), was reduced to ~800 FTEs, while build and implementation speeds increased significantly, even as errors were reduced.

↓ Step 3: Execute a proof of concept

To prove the value is real and validate the business case, the next step for a CIO is to greenlight a proof of concept. A good place to focus that is on incident processing. Companies that have implemented IPA for incident processing have been able to show cost savings of up to 30%. Thankfully, there are many work efforts (tickets) that can quickly be automated to serve as a proof of concept, such as incidents that are essentially a front end for already-automated processes based on mature APIs and tools, (password reset, setting up access for a new employee, ordering new equipment, etc.). A proof of concept requires:

Workshops with appropriate IT subject matter experts (SMEs) to understand all the steps and systems involved in a given process. This helps to identify where IPA can best be applied.
The thoughtful selection of an IPA platform. This decision has significant implications because platform capabilities, ambitions, and service providers vary, in some cases significantly. Some IPA platforms, for example, provide better integration capabilities, such as APIs that tie into existing systems. Others offer prepackaged or customizable bots, while some platforms are moving to provide AI capabilities even as others remain focused on process automation.
Obtaining the necessary approvals from IT (security guidelines, for example) and the business (access limitations and regulatory constraints)
Programming the bot(s) using iterative design techniques to ensure speed, accuracy, and scalability. At least one engineer needs to be assigned to manage the extensive testing that is a core element of the iterative design to ensure that the bot learns and adjusts based on live feedback.
Ongoing monitoring to document the results and ensure value capture

It is also useful to think of a pilot as a kickoff for internal IPA capability building, for example, by using a blend of internal and external developers to jump-start a future center of excellence (CoE). The team should become the home and engine of IPA learning.

↓ Step 4: Build IPA capabilities to scale

Realizing the full potential of IPA in IT requires a focus on building specific skills and capabilities, as well as adapting the new culture of the organization to eventually embed IPA at the heart of the IT organization. Typically, we see the most successful companies do three things:

Ramp up the success to new areas of IT: At this stage, the team is likely to move beyond basic help-desk level 0/1 incidents and pursue the automation of more advanced level-2 and level-3 tickets. The team should also expand beyond incidents and begin working on using IPA for monitoring, dashboarding, and analytics, moving from the help desk to the data center, the network, and even application-maintenance organizations. The long-term success of the automation program is contingent on how quickly the IPA bots are adopted within the IT organization. That depends on how effective leadership is in providing dedicated training and ongoing support, as well as building up a network of internal “reference cases.” The goal is to build on the successes to find new and more advanced use cases and opportunities within IT (as a precursor for generating demand across the wider organization). Providing incentives for IT employees in the form of bonus payments or recognition in competitions can be effective. At this point, the CIO needs to invest in capabilities that support scale, such as risk management and IT infrastructure management. These are different from those capabilities needed for pilots, which focus on getting the technology right, demonstrating value, convincing nonbelievers, etc. Leaders sometimes mix up up the two and underestimate what’s most important about each.
Getting the word out: With a solid foundation of experience and capabilities in place, the CIO can begin to actively position him/herself as both an advisor and enabler for the rest of the business. In practice, that means reaching out to leaders across various functions to inform them about the specific benefits of IPA, understanding their priorities and how to best implement and support the technologies ,and identifying potential security issues through bots IPA is by its nature disruptive. A CIO should have a clear sense of when IPA technologies will augment or replace human workers and put in place a program of clear communications and activities for each outcome.
Exploring advanced elements of IPA: While most IT organizations have focused on simple process automation (and to a lesser extent machine learning and natural-language processing), the future belongs to artificial intelligence and cognitive learning, which have the potential to manage complex IT tasks. Although still somewhat futuristic, the solutions are already emerging, and we expect them to rapidly mature over the next several years. But it takes time to build up the skills and experience needed to use AI effectively, in part because there is still a lot of confusion about what AI actually is. The only way to overcome that confusion is to start working on AI projects. Companies that are building up expertise in this area are developing data lakes, creating meaningful tags for that data, and then dedicating engineers to build and train algorithms to act on that data.

Notes

✓ Some of the AI tools that can be used to enable intelligent automation are IBM’s Watson, Salesforce Einstein, and Google Cloud Vision.
✓ Platforms that enable workflows that utilize these AI tools are called Intelligent Automation Platforms, Intelligent iPaaS, or Enterprise Automation Platforms.
✓ Cloud-based intelligent automation is distinct from RPA technology. The integrations are comprised of a series of cloud-based API connectors that can perform CRUD actions (mainly on NoSQL databases) and read / react to event triggers to create data corridors between disparate applications, using artificial intelligence and machine learning to enhance the power of the workflows. The cloud-based intelligent automation framework is based on business logic (trigger-events) that determine when, and if, actions in a sequence take place between applications.
✓ Integration Platform as a Service (iPaaS) is a suite of cloud services enabling development, execution and governance of integration flows connecting any combination of on premises and cloud-based processes, services, applications and data within individual or across multiple organizations.

Market glance

Content contextualization

Process-centric application platforms

Robotic Process Automation

Integration and API management

Decision automation and optimization

Process intelligence and planning

Everest Group Matrix Assessment

2023
Everest Group Matrix Assessment 2023

2022
Everest Group Matrix Assessment 2022

2021
Everest Group Matrix Assessment 2021

Conclusion

The value of intelligent automation in the world today, across industries, is unmistakable. With the automation of repetitive tasks through IA, businesses can reduce their costs as well as establish more consistency within their workflows. The COVID-19 pandemic has only expedited digital transformation efforts, fueling more investment within infrastructure to support automation. As remote work also surges, roles will continue to evolve. Individuals focused on low-level work will be reallocated to implement and scale these solutions as well as other higher-level tasks. Middle managers will need to shift their focus on the more human elements of their job to sustain motivation within the workforce. Automation will expose skills gaps within the workforce, and employees will need to adapt to their continuously changing work environments. Middle management can also support these transitions in a way that mitigates anxiety to ensure that employees remain resilient through these periods of change. Intelligent automation is undoubtedly the future of work, and companies that forgo adoption will find it difficult to remain competitive in their respective markets.